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I.  Introduction 

A  central  problem  in  the  study  of  vision  is  the  deter¬ 
mination  of  the  cellular  neural  mechanisms  underlying 
behaviorally  relevant  functions.  Unfortunately,  no  single 
approach  or  technique  has  proved  sufficient  by  itself  to 
unravel  all  the  complexities  involved  and  provide  a  complete 
understanding  of  visual  behavior.  Precise  assessment  of 
behavioral  performance  with  well-defined  tasks  in  the  intact 
animal  can  yield  a  quantitative  description  of  visual 
behavior  but  any  inferences  concerning  causal  neural  mecha¬ 
nisms  must  remain  speculative  since  the  central  nervous 
system  of  any  species  of  interest  is  too  complicated  for  a 
unique  mechanism  to  be  identified.  As  Dean  describes 
in  a  discussion  of  the  inferotemporal  cortex  elsewhere  in 
this  book,  the  same  limitation  applies  to  behavioral  studies 
of  lesioned  animals.  At  the  other  end  of  the  spectrum 
of  approaches,  neuronal  activity  can  be  measured  directly 
at  some  level  of  the  visual  pathway  by  fracturing  the 
integrity  of  the  central  nervous  system  with  anesthesia, 
drugs  or  surgery, and  with  the  application  of  techniques  from  the 
armamentarium  of  modern  neurobiology,  specific  cellular 
mechanisms  can  often  be  identified.  However,  as  several 
chapters  in  this  book  emphasize,  visual  processing  in  the 
central  nervous  system  is  distributive,  that  is  to  say  there 
are  multiple  visual  systems,  so  that  a  particular  candidate 
mechanism  may  be  irrelevant  to  the  behavior  of  interest  or 
may  operate  differently  in  the  intact  animal.  However, 
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converging  evidence  from  different  approaches  to  the  study 
of  the  visual  system  can  offer  an  effective  strategy. 

The  primate  striate  cortex  provides  a  model  system  to 
illustrate  such  a  strategy.  Considerable  information  has 
accumulated  concerning  the  anatomy  and  physiology  of  the 
striate  cortex  and  antecedent  structures  in  the  retino- 
geniculo-cortical  pathway.  In  contrast  to  the  afferent 
visual  pathways  of  other  species  (see  for  example  the  afferent 
pathways  in  the  cat  as  illustrated  in  Fig.  1  in  the  chapter 
by  Berkley  and  Sprague)  the  striate  cortex  in  Old  World 
anthropoids  is  the  major  if  not  exclusive  recipient  of  fine- 
grain  visual  information.  For  Old  World  anthropoids  the 
cross-species  similarity  assumption  appears  valid,  so  that 
behavioral  and  neural  data  obtained  for  one  species  can  be 
assumed  to  hold  for  all  others.  To  explain  visual  perfor¬ 
mance  in  general,  a  reasonably  complete  if  not  exhaustive 
knowledge  of  the  central  nervous  system  is  required,  but 
for  optimal  performance  such  as  that  in  a  visual  acuity  task 
we  need  only  focus  on  the  relevant  mechanisms  that  provide 
constraints  on  optimality.  Strong  evidence  places  the  locus 
of  such  constraints  in  the  striate  cortex,  and  the  quantifi¬ 
cation  of  striate  receptive  fields  and  visual  performance  on 
acuity  tasks  makes  possible  the  development  of  a  computational 
theory  relating  causal  neural  mechanisms  to  behavior.  In 
this  chapter  a  combined  approach  of  neural  and  behavioral 
analyses  and  mathematical  synthesis  is  applied  to  the  role 
of  the  striate  cortex  in  visual  acuity.  First  those  critical 
features  of  the  primate  visual  system  that  are  pertinent  to  the 
discussion  are  reviewed. 
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II .  Primate  Anatomy  and  Physiology 

Existing  Old  World  anthropoids  which  includes  a  range  of 
species  from  monkey  (e.  g.  macaca  mulatta)  to  man  (e.  g.  homo 
sapiens)  exhibit  a  number  of  special  evolutionary  adaptations  in 
their  visual  systems.  These  species  most  probably  having  evolved 
from  nocturnal,  solitary  visual  predators  (Cartmill,  1974; 

Polyak,  1957)  into  diurnal,  social  frugivores  or  omnivores 
possess  frontally  directed  eyes  with  large  degree  of 
binocular  overlap  of  150  °,  high  central  visual  acuity  (Blakemore, 
1970) ,  well  developed  trichromacy  (DeValois  and  Jacobs,  1968) 
and  stereopsis  to  baffle  natural  camoflage  and  promote  their 
precise  eye-hand  coordination,  excellent  visuospatial  memory 
(Menzell,  1973)  and  acute  perception  of  visual  social  cues 
(Savage-Rumbaugh,  Rumbaugh  and  Boysen,  1978) .  Subserving  these 
behavioral  adaptations  there  are  a  number  of  neural  adaptations 
not  all  of  which  are  known  or  understood  at  the  present  time. 

A.  Foveal  specialization 

The  retina  of  Old  World  anthropoids  and  indeed  of  a  number 
of  other  primates  possesses  a  specialization  corresponding  to 
the  center  of  visual  fixation,  the  fovea.  As  in  all  vertebrate 
retinas  the  photoreceptors  are  located  at  the  inner  margin  of 
the  retina  facing  away  from  the  light.  However,  in  the  primate 
retina  at  the  region  corresponding  to  the  center  of  gaze  the 
proximal  neurons  in  the  inner  nuclear  layer  (amacrine  and 
ganglion  cells)  are  displaced  laterally  forming  the  foveal 
pit  and  allowing  light  more  direct  access  to  the  transduction 
elements.  Reduction  in  light  scattering  is  also  achieved  by 
avoidance  of  the  foveal  region  by  the  blood  vessels  on  the 
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vitreal  surface  of  the  retina  leaving  intact  only  the  choroidal 
circulation.  Associated  with  the  ciliary  stocks  of  the  photo¬ 
receptors  in  the  fovea  (cones)  is  the  dense  yellow  pigment,  the 
macular  pigment,  which  serves  to  absorb  shortwave  light  and  reduce 
the  chromatic  abberation  inherent  in  the  simple  lens  system  of  the 
eye.  Flicker  photometry  measurements  (Wooten,  Thawley  and  Knoblauch, 
1980)  indicate  that  the  macular  pigment  is  densest  in  the  center  of 
the  fovea  with  its  distribution  dropping  off  sharply  outside  the 
fovea  in  parallel  with  the  cone  distribution.  Microspectrophotometric 
measurements  of  cone  outer  segments  have  revealed  a  basis  for  tri¬ 
chromatic  color  vision  in  the  differing  absorption  spectra  with 
X  of  430nm,  535nm,  565nm  (Marks,  Dobell  and  MacNichol,  1964; 

Wald  and  Brown,  1965;  Bowmaker,  Dartnall  and  Mollon,  1980) 
described  by  a  modified  vitamin  A^  nomogram  (Ebrey  and  Honig, 

1977).  Although  the  center  of  the  fovea  is  rod-free  the  dense 
packing  of  the  cones  distorts  the  outer  segment  into  slender  rod¬ 
like  shape  (Dowling  and  Boycott,  1966)  from  their  normal  pyramidal 
shape  found  in  the  periphery.  Although  rods  and  cones  probably 
do  not  differ  significantly  in  intrinsic  sensitivity  (Fain  and 
Dowling,  1973)  they  do  exhibit  substantially  different  saturation 
characteristics  (Boynton  and  Whitten,  1970).  The  convergence 
necessitated  by  the  existence  of  200  million  photoreceptors 
but  only  1  million  optic  nerve  fibers  is  not  uniform  but 
graded  across  the  retina  with  the  central  region  receiving  a 
disproportionate  share  as  reflected  in  the  magnification 
factor  in  the  retinotopic  map  of  the  dorsal  lateral  geniculate 
(Malpeli  and  Baker,  1975) .  In  contrast  to  other  vertebrate 
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species  the  retinal  circuitry  of  Old  World  anthropoids  is 
simple  containing  relatively  few  classes  of  amacrine  cells 
and  correspondingly  the  retinal  ganglion  cells  fall  into 
relatively  few  classes.  Following  terminology  developed  in 
the  cat  retina,  ganglion  cells  have  been  classified  into  three 
classes  termed  X,  Y  and  when  specified,  W  (DeMonasterio 
and  Gouras,  1975;  Schiller  and  Malpeli,  1977;  DeMonasterio, 
1978a) .  X  cells  have  small  receptive  fields,  summate  linearly 
across  these  fields,  frequently  have  a  sustained  response 
to  standing  contrast  and  are  found  predominantly  in  the 
foveal  region.  Y  cells  have  large  receptive  fields,  summate 
nonlinearly  across  these  fields,  mainly  exhibit  a  transient 
response  to  standing  contrast  and  are  found  predominantly  in 
the  periphery.  In  quantitative  terms  the  central  fovea  (1°) 
yields  90%  X  cells  and  10%  Y  cells,  but  in  the  near  periphery 
(11-20°)  only  45-70%  X  cells  and  30-50%  Y  cells  (Schiller  and 
Malpeli,  1977;  DeMonasterio,  1978a).  W  cells,  clearly  not  a 
homogeneous  class,  lack  well  defined  center-surround  organi¬ 
zation,  are  distributed  across  the  retina  but  mainly  outside 
the  foveal  region,  and  project  directly  to  the  superior 
colliculus  rather  than  throuqh  the  geniculate  with  some 
exceptions  (Wiesel  and  Hubei,  1966;  Dreher,  Fukada  and  Rodieck, 
1-76;  DeMonasterio,  1978b)  providing  in  the  primate  (in 
contrast  to  other  mammals)  a  complete  representation  of  only  the 
contralateral  half  of  the  visual  field.  The  actual  proportion 
of  the  retinal  ganglion  cell  distribution  consisting  of  W  cells 
is  difficult  to  estimate  since  the  recording  techniques  tend 
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to  sample  from  the  larger  or  more  responsive  X  and  Y  cells. 
Rodieck  (1979)  estimates  that  possibly  50%  of  cat  retinal 
ganglion  cells  are  W  cells,  so  that  in  monkey  the  proportion 
may  be  larger  than  the  10%  encountered  experimentally.  The 
retinal  segregation  into  separate  functional  classes  undoubtedly 
has  adaptive  significance  since  it  is  retained  at  higher 
levels  in  the  visual  system. 

B.  Functional  segregation  in  the  dLGN 

The  axons  of  the  retinal  ganglion  cells  form  the  optic 
nerve  which  decussates  at  the  optic  chiasm  before  reaching 
its  targets  in  the  dorsal  region  of  the  lateral  geniculate 
of  the  thalmus  and  the  superior  colliculus  in  the  brain  stem. 
Each  hemi-retina  has  a  geniculo-cortical  target  in  a  separate 
hemisphere  except  for  a  1°  vertical  strip  along  the  midline 
and  a  few  ganglion  cells  around  the  rim  of  the  fovea  'Stone, 
Leicester  and  Sherman,  1973;  Bunt,  Minckler  and  Johanson,  1977). 
The  primate  dLGN  is  a  6-layered  structure  receiving  in 
precise  retinotopic  register  the  inputs  from  the  two 
retinas  and  maintaining  them  in  separate  laminae  with  a 
discrete  functional  organization  as  well  (Schiller  and  Malpeli, 
1978;  Dreher,  Fukada  and  Rodieck,  1976).  The  4  parvocellular 
layers  receive  X  cell  input  and  have  cells  with  concentric 
color-opponent  receptive  field  organization  (R+G-  or  G+R-) ; 
the  blue-selective  on-cells  are  found  predominantly  in  the 
ventral  pairs  of  parvocellular  layers.  The  two  magnocellular 
layers  receive  Y  cell  input  and  have  cells  that  are  concentri¬ 
cally  organized  but  as  might  be  expected  have  spectrally 
broad  band  or  achromatic  receptive  fields. 
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C.  Cortical  organization 

In  contrast  to  other  mammalian  species  (e.q.  see  the 
chapter  by  Berkley  and  Sprague  for  a  discussion  of  the 
multiple  and  parallel  input  pathways  into  the  cortex  of  the 
cat)  the  dLGN  of  catarrhine  monkeys,  apes  and  man  forms  the  major  if 
not  exclusive  input  into  the  cortex,  and  that  entry  is  confined  to 
the  striate  cortex  (Area  17  of  Brodman)  as  shown  in  rig.  1. 

Insert  Fig.  1  about  here 


Moreover  the  functional  segregation  present  in  the  geniculate 
is  maintained  though  less  strictly  in  this  initial  cortical 
stage.  The  axons  of  the  X  cells  of  the  parvocel lular  layers 
terminate  mainly  in  layer  4A  and  layer  4C6  of  this  6-layered 
koniocortical  region  (Hubei  and  Wiesel,  1972).  The  field 
of  termination  is  the  spiny  stellate  neurons  which  project 
to  layer  3B  and  5A  as  well  as  within  4C  (Lund,  1973;  Lund 
and  Boothe,  1975).  The  axons  of  the  Y  cells  emerging  from 
the  rnagnocellular  layers  project  mainly  to  layer  4Ca  (Hubei 
and  Wiesel,  1972);  however,  they  also  bifurcate  and  terminate 
in  the  superior  colliculus  (Schiller,  MalpeJ i  and  Schein, 
1979).  The  field  of  termination  is  the  spiny  stellate  neurons 
which  contribute  to  laminae  4B  and  5A  but  also  distribute 
widely  within  4Ca  (Lund,  1973;  Lund  and  Boothe,  1975). 

Both  anatomical  evidence  based  on  Golgi  staining  (Lund  and 
Boothe,  1975)  and  electrophysiological  evidence  based  on 
selective  anesthetization  of  a  specific  geniculate  lamina 
(Schiller,  Malpeli  and  Schein,  1979)  suggest  that  the  input 
from  the  parvocel lular  and  rnagnocellular  geniculate  neurons 
remains  largely  segregated  with  X  cells  being  the  dominant 
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influence  for  cortical  neurons  with  cell  somas  in  the  upper 
or  supragranular  layers  and  Y  cells  being  the  major  influence 
for  the  inf ragranular  layers.  Such  a  distinction  has  a 
definite  functional  significance  since  the  efferent  pro¬ 
jections  of  the  two  regions  are  quite  different. 

Area  17  or  Vl  projects  and  receives  reciprocal  projec¬ 
tions  from  both  cortical  and  subcortical  targets  with  the 
cortical  targets  receiving  efferents  predominantly  from  the 
supragranular  layers  and  the  subcortical  targets  receiving 
efferents  from  the  inf ragranular  layers  (Lund,  Lund, 
Hendrickson,  Bunt  and  Fuchs,  1975). 

The  pyramidal  cells  in  the  upper  regions  of  layer  6  project 
back  inhibitorily  to  the  parvocellular  layers  of  the  dLGN 
but  the  pyramidal  cells  in  the  lower  portion  of  layer  6 
project  back  to  the  magnocellular  layers  (Marocco,  McClurkin 
and  Farooqui,  1979) .  The  pyramidal  cells  in  layer  5  project 
retinotopically  to  both  superior  colliculus  and  pulvinar 
but  receive  a  reciprocal  projection  to  layer  2  only  from 
pulvinar  which  also  projects  widely  to  the  rest  of  the  visual 
field  (Diamond,  1979) .  However,  the  projection  from  the 
pulvinar  to  the  more  rostral  regions  of  cortex  does  not 
contribute  to  the  spatial  selectivity  of  the  cells  in  these 
regions  (Gross  and  Mishkin,  1977).  Both  the  pyramidal  and 
stellate  cells  in  layer  4A  which  exhibit  strong  directional 
selectivity (Dow,  1974)  as  well  as  the  largest  pyramidal  cells 
in  the  upper  portion  of  layer  6  project  to  MT  which  in  turn 
sends  a  reciprocal  projection  to  layers  4A  and  6.  This  circuit 
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may  contribute  to  visually  guided  movement  rather  than  object 
vision  since  MT  sends  a  heavy  projection  to  the  pontine 
nucleus  which  then  relays  the  information  to  the  cerebellum 
(Glickstein,  1979)  .  The  pyramidal  cells  in  the  supragranular 
layers  project  transcortically  to  V2  and  V3  both  ipsilaterally 
and  near  the  V1/V2  border  contralaterally  as  well;  these 
regions  reciprocate  with  a  projection  to  layer  1  (Spatz  and 
Tigges,  1972;  Wong-Riley,  1979).  In  addition  there  is  a 
recently  discovered  third  cortical  projection  site  in  the  most 
caudal  portion  of  the  interparietal  sulcus  (see  chapter  by 
Ungerleider  and  Mishkin  for  a  review  of  the  evidence  for  the 
site) .  The  cortico-cortical  pathways  from  the  supragranular 
layers  conveying  the  transformed  input  from  the  parvocellular 
X  cells  are  undoubtedly  the  critical  ones  for  pattern  vision. 

Since  in  primates  but  not  other  mammals  Vl  is  the 
dominant  if  not  exclusive  entry  site  of  visual  information 
into  the  cortex,  the  supragranular  layers  of  Vl  as  indicated 
in  Fig.  1  are  the  gateway  to  an  arena  of  rich  cortical  inte¬ 
gration.  The  terminal  target  regions  in  the  visual  projection 
field  have  been  implicated  in  different  classes  of  visual 
behavior:  (1)  MT  could  play  a  role  in  visually  guided 

movement  (Glickstein,  1979);  (2)  Area  TE  of  von  Bonin  and 
Bailey  on  the  inferior  convexity  of  the  temporal  lobe  appears 
to  mediate  the  acquisition  of  complex  visual  discriminations 
(Gross  and  Mishkin,  1977;  see  chapter  by  Dean  in  this  book); 

(3)  Area  8  in  the  frontal  lobe  serves  as  a  second  control 
center  for  saccadic  eye  movements  in  addition  to  the  superior 
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colliculus  (Schiller,  True  and  Conway,  1979);  (4)  Area  7 
in  the  parietal  lobe  appears  to  direct  visual  attention  to 
objects  in  extrapersonal  space  (Lynch,  Mountcastle,  Talbot 
and  Yin,  1977;  Mountcastle,  1978;  see  chapter  by  Ungerleider 
and  Mishkin  in  this  book  for  evidence  from  lesion  studies 
implicating  this  region  in  object  localization).  However, 
these  more  rostral  regions  of  the  visual  system  may  be 
irrelevant  or  at  least  redundant  for  certain  basic  pattern 
detection  tasks  since  all  regions  of  the  cerebral  cortex 
project  to  the  basal  ganqlia,  the  major  motor  control  system. 
In  particular,  visual  acuity  is  unimpaired  by  either  infero- 
temporal  lesions  (Cowey  and  Weiskrantz,  1967;  Blake,  Jarvis 
and  Mishkin,  1977;  Weiskrantz,  1980)  or  by  parietal  lobe 
ablation  (see  chapter  by  Ungerleider  and  Miskin  in  this  book) 
which  suggests  that  some  relatively  simple  detection  mechanism 
applied  to  the  output  of  Area  Vl  may  be  sufficient  for  high 
level  performance  in  a  visual  acuity  task.  It  is  of  interest 
then  to  consider  how  visual  information  is  processed  in  VI 
and  represented  in  the  population  profile  of  its  efferents. 
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III .  Cortical  Processing  and  Representation  in  Vl 

In  order  to  analyze  the  contribution  of  VI  it  is  necessary 
first  to  consider  visual  information  that  remains  essentially 
invariant  so  as  to  define  more  clearly  the  intrinsic  processes. 
A.  Invariants  of  retinal  and  geniculate  origin 

The  receptive  fields  of  individual  neurons  recorded  in 
the  supragranular  layers  of  striate  cortex  preserve  invariant 
or  only  slightly  transformed  several  properties  encoded  in 
more  distal  portions  of  the  visual  pathway.  The  minimum 
discharge  receptive  fields  are  discrete  with  excitatory  cores 
only  slightly  greater  in  size  than  those  of  the  dLGN  (Schiller, 
Finlay  and  Volman,  1976a)  maintaining  approximately  the  same 
magnification  factor  as  in  the  geniculate  (Hubei  and  Wiesel , 
1974)  but  with  a  somewhat  less  precise  retinotopic  organization. 
Light-  and  dark-adaptation  have  their  major  locus  of  control 
in  the  retina  so  that  for  example  the  inverse  relation 
between  discharge  latency  and  the  luminance  of  the  background 
or  target  is  determined  up  to  an  additive  constant  by  retinal 
and  in  the  dark-adapted  case  more  likely  by  photoreceptor 
mechanisms  (Mansfield,  1976b;  Mansfield  and  Daugman,  1978). 
Object  brightness  which  appears  to  be  encoded  by  relative 
impulse  density  is  established  for  isolated  targets  at  the 
retinal  ganglion  cell  level  in  a  relation  that  is  initially 
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linear  up  to  one  log  unit  above  the  background  level  but 
nonlinear  at  higher  levels  (Mansfield,  1976a, b;  1975).  In 
the  dark-adapted  case,  simple  reaction  time  and  the  latency 
of  cortical  evoked  potentials  decrease  as  the  inverse  cube  root 
of  target  luminance  approaching  asymptotically  different 
but  constant  values.  In  the  dark-adapted  case,  retinal 
ganglion  cells,  lateral  geniculate  neurons  and  supragranular 
striate  cells  (although  only  10%  of  the  population)  increase 
their  average  discharge  rate  as  the  cube  root  of  target 
luminance,  a  relation  also  found  with  numerical  magnitude 
estimates  in  psychophysical  tasks.  However,  the  brightness 
or  lightness  of  targets  in  more  complex  scenes  is  influenced 
by  large  range  spatial  interactions  (e.  g.  Land  and  McCann, 

1971)  which  undoubtedly  reflect  central  convergence.  Such  global 
interactions  of  lightness  may  underlie  object  color  perception 
but  local  spectral  and  achromatic  differences  are  present 
more  distally.  The  majority  of  neurons  in  the  supragranular 
layers  of  the  foveal  projection  region  maintain  a  differential 
sensitivity  to  wavelength  (Poggio,  Baker,  Mansfield, 

Sillito  and  Gregg,  1975;  Crawford,  Fagan,  Borchert,  Heston 
and  Marc,  1979)  but  in  the  representations  of  the  more 
peripheral  regions  the  proportion  drops  sharply  relative  to 
that  of  the  geniculate  indicating  some  convergence.  In 
addition,  the  constant  criterion  thresholds  of  the  monkey 
spectrally  broad  band  cells  in  both  the  foveal  and  parafoveal 
projection  regions  reflect  the  same  balanced  input  from  the  three 
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classes  of  cone  photoreceptors  exhibited  by  behavioral 
measures  of  luminance  threshold  in  the  intact  animal  (Poggio 
et  al,  1975).  On  the  other  hand,  four  attributes  of  cortical 
receptive  fields  represent  intrinsic  contributions  of 
striate  cortex. 

B.  Intrinsic  cortical  transformations 

There  are  four  major  transformations  carried  out  upon 
the  incoming  visual  information  by  the  synaptic  circuitry  of 
VI  and  reflected  in  the  discharge  patterns  of  efferent 
neurons:  (1)  binocular  convergence;  (2)  spatial  frequency 

filtering;  (3)  pattern  specific  adaptation;  (4)  orientation 
and  direction  selectivity.  For  each  visual  hemifield  Vl 
represents  the  first  site  of  binocular  convergence.  With 
increasing  distance  from  layer  4  the  proportion  of  binocularly 
driven  neurons  increases  in  a  manner  suggesting  independence 
of  connectivity  for  orientation  selectivity  and  ocular 
dominance  (Poggio,  1972).  Although  the  degree  of  orientation 
preference  disparity,  i.  e.  the  amount  to  which  the  axis 
of  orientation  differs  in  the  two  eyes  is  small  in  primates, 
estimated  to  be  4°  in  the  macaque  monkey  compared  with  9° 
in  the  cat  (Mansfield,  Ronner  and  Daugman,  1980) ,  there  is 
also  a  binocular  disparity  in  the  retinal  locus  of  receptive 
fields  that  probably  serves  a  role  in  local  stereopsis. 

Poggio  and  Fischer  (1977)  found  that  in  the  foveal  striate 
projection  region  where  receptive  fields  were  small  the 
majority  of  neurons  were  sensitive  to  location  of  stimuli 
in  depth  with  the  maximum  discharge  or  shift  in  discharge 
occurring  within  +  0.4°  of  disparity  but  the  total  range 
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of  binocular  interaction  could  extend  to  more  than  +1  of 
disparity.  A  second  transformation  that  takes  place  is 
that  the  spatial  frequency  filtering  present  in  the  geniculate 
neurons  becomes  sharper  (Poggio,  Doty  and  Talbot,  1977; 

DeValois,  Albrecht  and  Thorell,  1978).  The  third  transformation 
is  the  adaptation  or  habituation  of  neural  discharges  to 
specific  patterned  stimuli  (Mansfield,  Ronner  and  Daugman, 

1978)  a  feature  not  observed  in  the  lateral  geniculate 
(DeValois  et  al ,  1978;  Wiesel  and  Hubei,  1966;  Schiller  and 
Malpeli,  1978)  but  whose  appearance  at  the  cortical  level 
reflects  the  initiation  of  a  temporal  filtering  mechanism 
for  visual  salience  and  novelty.  A  fourth  and  major  transfor¬ 
mation  is  that  instead  of  isotropic,  concentrically  organized 
receptive  fields  such  as  those  found  in  the  dLGN  the  majority 
of  striate  neurons  in  the  supragranular  layers  exhibit  an 
orientation  preference  and  are  directionally  selective 
(Hubei  and  Wiesel,  1968;  Poggio,  1972;  Dow  and  Gouras,  1973; 

Dow,  1974;  Poggio  et  al,  1975;  Schiller,  Finlay  and  Volman, 
1976a, b;  Mansfield  et  al,  1980) .  Trom  recent  experiments 
quantitative  estimates  can  be  made  of  the  population  response 
characteristics  in  the  efferent  projection  from  VI. 

C.  Quantitative  measurements  of  receptive  field  properties 
In  order  to  predict  behavioral  performance  it  is 
necessary  first  to  know  in  quantitative  detail  the  excit¬ 
ability  profiles  of  the  receptive  fields  of  the  efferent 
neurons  in  the  supragranular  layer  and  secondly  to  understand 
at  least  to  a  first  approximation  how  the  neural  information  is 
utilized  in  a  particualr  behavioral  task. 
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For  the  first  problem  the  main  obstacle  is  that  of 
sample  bias  when  recording  from  only  a  small  proportion  of 
neurons  in  the  layers.  One  source  of  sample  bias  arises 
from  the  columnar  organization  for  the  attributes  of  orien¬ 
tation  selectivity  and  ocular  dominance  (Hubei  and  Wiesel,  1963). 
The  invariance  of  particular  receptive  field  properties  with 
cortical  depth  is  a  general  principle  of  neocortical  organ- 
tation  (Mountcastle,  1957;  Mountcastle,  1978)  probably  reflect¬ 
ing  the  conservation  of  genetic  information  required  for 
synaptic  organization  of  the  cortex  using  modular  subunits. 

From  the  experimenter's  point  of  view,  however,  microelectrode 
penetrations  normal  or  nearly  normal  to  the  pial  surface 
encounter  neurons  whose  receptive  field  properties  are  not 
independent;  oblique  penetrations  encounter  similar  problems 
of  statistical  dependence  when  the  attributes  like  orientation 
selectivity  exhibit  sequential  regularity  (Hubei  and  Wiesel, 

1974) .  One  strategy  for  obtaining  statistical  independence 
is  to  count  as  a  sample  only  a  single  neuron's  receptive 
field  (e.g.  the  first  neuron  encountered  that  could  be 
studied)  in  a  given  penetration  and  to  separate  penetrations 
by  at  least  some  fixed  distance  (e.g.  at  a  separation  of 
500  microns  the  mean  serial  correlation  coefficient  for 
orientation  preference  has  dropped  to  chance  levels) 

(Mansfield,  1974;  Mansfield  et  al,  1980).  A  second  potential 
source  of  sample  bias  with  microelectrode  penetrations  is 
that  the  neurons  with  large  cell  somas  will  be  more  heavily 
weighted  since  their  action  potentials  will  be  more  easily 
isolated  (Humphrey  and  Corrie,  1978).  However,  since  the 
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neurons  of  interest  are  the  pyramidal  cells  in  layers  2-3 
rather  than  the  stellate  cells  such  a  bias  will  aid  in  selecting 
the  population  of  interest.  A  third  potential  source  of 
bias  is  the  stochastic  nature  of  the  neuronal  discharge; 
however,  multiple  and  randomized  stimulus  presentation 
can  provide  objective,  stable  measures  (Stryker  and  Sherk, 

1975;  Schiller,  Finlay  and  Volman , 1976a ;  Mansfield  et  al,  1980). 
A  fourth  potential  source  of  bias  is  that  a  particular  choice 
of  measure  for  neuronal  discharge  may  not  be  relevant  for 
further  processing  by  the  more  rostral  portions  of  the  visual 
system;  however,  integrated  discharge  (pulse  number)  for 
brief  stimulus  presentations  is  a  probable  candidate  code. 

) )  Excitability  profiles 

The  majority  of  the  receptive  fields  in  the  supra- 
granular  layers  of  primate  Vl  have  discharge  characteristics 
resembling  X  cells  in  the  geniculate  but  spatial  character¬ 
istics  of  complex  cells,  i.e.  a  single  activating  region 
(Hubei  and  Wiesel ,  1968;  Poggio  et  al,  1975;  Schiller  et  al, 
1976a ;Poggio  et  a^L,  1977;  Mansfield  et  al ,  1980).  No  receptive 
field  type,  e.g.  simple  cells,  is  distinguishable  on  the 
basis  of  receptive  field  size  or  breadth  of  orientation 
tuning  (Schiller  et  a_l,  1976;  Mansfield  ot  a  1  ,  1980).  At 
the  descriptive  level  a  central  excitatory  zone  with  flanking 
silent  cortical  inhibition  of  cortical  origin  is  a  first 
appro:;  j.i.iat  ion . 

2 )  Orientation  tuning 

As  with  orientation  tuning  curves  in  the  cat  when  they 
have  been  examined  in  quantitative  detail  (Henry,  Dreher  and 
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Bishop,  1974) ,  a  symmetric  bell-shaped  curve  provides  an  excellent 
fit  to  primate  data  as  illustrated  in  Fig.  2  for  a  binocular  cell. 


Insert  Fig.  2  about  here 

The  shape  of  the  orientation  tuning  curve  does  not  change  systemati¬ 
cally  with  orientation  preference  or  with  retinal  eccentricity  at 
least  between  the  central  foveal  projection  (0-2°)  (Mansfield  et  al , 
1980)  and  parafoveal  region  (3-6°)  (Schiller  et  al,  1976b;  Mansfield 
et  al,  1980)  where  it  has  been  examined.  However,  as  illustrated  in 
Fig.  3  the  distribution  of  tuning  bandwidths  (half-width  at  half-height) 


Insert  Fig.  3  about  here 

is  broad  but  skewed  toward  the  narrowly  tuned  neurons  with  a  median 
near  24°,  a  value  that  is  consistent  across  electrophysiological 
experiments  (Schiller  et  al,  1976;  Mansfield  et  a^,  1980).  One  strik¬ 
ing  difference  illustrated  in  Fig.  4  between  foveal  and  parafoveal 


Insert  Fig.  4  about  here 

projection  regions  is  that  in  the  parafoveal  projection  region  all 
orientations  are  equally  represented  (Hubei  and  Wiesel,  1968;  Finlay, 
Schiller  and  Volman,  1976)  but  in  the  foveal  projection  region  the 
horizontally  and  vertically  selective  receptive  fields  are  over¬ 
represented  (Mansfield,  1974;  Mansfield  and  Ronner,  1978;  Mansfield 
et  al,  1980) .  The  orientation  anisotropy  is  probably  innate  and  is 
not  confined  to  primates  (e.g.  for  the  cat:  Pettigrew,  Nikara  and 
Bishop,  1968;  Levanthal  and  Hirsch,  1977;  Kennedy  and  Orban,  1979; 
Orban  and  Vandenbussche,  1979) . 


•T  -  *ti  _•  fit 
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3)  Size  or  spatial  frequency  tuning 

The  size  of  receptive  fields  in  the  supragranular  layers  changes 
with  two  gradients.  First  the  average  size  of  receptive  field 
increases  with  retinal  eccentricity  (Hubei  and  Wiesel,  1974).  Second, 
the  size  of  receptive  field  increases  with  distance  from  layer  4,  the 
main  input  layer.  So  that  at  each  location  in  the  retinotopic  pro¬ 
jection  there  is  a  distribution  of  receptive  field  sizes.  One  method 
of  quantifying  the  degree  of  size  tuning  of  the  cell  is  to  measure 
the  response  of  the  cell  to  grating  targets.  When  displayed  along 
a  log  spatial  frequency  axis  the  response  curves  assume  a  shape  that 
is  approximately  1-2  octaves  wide  at  half-maximal  response  (Schiller, 
Finlay  and  Volman,  1976c;  Poggio  et  a^l,  1977)  or  at  half-maximal 
sensitivity  for  a  constant  criterion  response  (DeValois,  Albrecht  and 
Thorell,  1978).  The  available  samples  of  neurons  from  the  supragranular 
layers  for  which  optimal  spatial  frequency  has  been  measured  are 
relatively  small  but  an  approximation  to  the  distribution  is  shown 
in  Fig.  5.  An  additional  factor  is  the  degree  to  which  neurons  with 


Insert  Fig.  5  about  here 

different  optimal  spatial  frequencies  respond  at  their  optimal  fre¬ 
quency  to  contrast  differences;  however,  there  is  essentially  no 
quantitative  data  that  can  be  brought  to  bear  on  this  question. 

4)  Pattern-specific  adaptation 

A  common  observation  is  that  neurons  in  the  supragranular  layers 
are  difficult  to  activate  repeatedly  and  habituate  easily  (Wurtz, 
1969) .  The  decrement  in  response  is  exponential  with  time  and 
depends  upon  the  cell's  response,  i.e.  non-optimal  stimulation  does 
not  produce  habituation  (Mansfield,  Simmons  and  Daugman,  1979).  The 
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exponential  time  constants  vary  across  cells  and  are  smaller  for 
adaptation  than  for  recovery  in  which  case  the  average  value  is 
on  the  order  of  10  sec.  Direct  synaptic  activation  of  the  supra- 
granular  pyramidal  cells  produces  habituation  in  vitro  (Mansfield  and 
Simmons,  1979) that  follows  a  similar  time  scale  for  recovery. 

5)  Quantitative  model  of  population  response  profile 

In  order  to  develop  a  computational  theory  of  the  function 
of  the  supragranular  layers  of  VI  based  on  the  population  response 
profile,  we  need  to  consider  first  the  excitation  profile  of  a 
typical  cortical  receptive  field  in  the  supragranular  layers;  second, 
the  local  and  global  distribution  of  receptive  field  parameters; 
and  finally  a  candidate  algorithm  for  combining  the  neural  output 
from  VI  to  produce  a  response. 

i)  Cortical  receptive  field  excitation  profile 

The  experimental  studies  reviewed  above  indicate  that  the 
cortical  receptive  fields  in  the  supra-granular  layers  are 
shape-invariant  with  changes  in  preferred  orientation  and 
scale  multiplicatively  in  the  frequency  domain.  As  a 
simplified  but  instructive  approach  to  the  question  of  the 
construction  of  a  cortical  receptive  field,  we  can  consider 
how  the  receptive  fields  at  successive  levels  in  the  primate 
visual  system  are  generated.  For  convenience,  the  on-center 
retinal  ganglion  X  cell  will  be  used  as  an  example.  The 
spatial  domain  weighting  function  describing  the  contributions 
of  the  photoreceptor  input  to  the  cell  can  be  reasonably  well 
approximated  by  a  circularly  symmetric  Gaussian  distribution, 
the  expected  limiting  distribution  as  the  number  of  photo- 
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receptors  becomes  large  assuming  isotropic  and  globally  random  connec¬ 
tivity.  However,  lateral  inhibition  is  also  contributed  by  the  retinal 
circuitry  and  can  be  represented  by  the  simplest  isotropic  form  of 

2 

spatial  differentiation,  the  two-dimensional  Luplacian  operator,  v  , 
defined  as 


2 

3 

or  more  simply  as  2  for  circularly  symmetric  spatial  functions 

3r 

in  polar  coordinates.  The  resulting  excitation  profile,  E(r),  for 
the  retinal  ganglion  cell  is  given  by  the  equation 

-  2  r  2 

v  G  =  -  ^T)e‘(I)  (5.2) 

a  a 

respresenting  central  excitation  with  surround  inhibition  as  depicted 
in  Fig.  6.  The  profile  is  essentially  similar  to  the  empirically 

Insert  Fig.  6  about  here 

derived  difference  of  Gaussians  used  by  Enroth-Cugeli  and  Robson  (1966) 
to  describe  the  structure  of  cat  retinal  ganglion  cells.  At  the  next 
level  in  the  parvocel lular  layers  of  the  lateral  geniculate,  the 
principal  cells  receive  excitatory  input  from  probably  a  very  small 
number  of  retinal  ganglion  cells  of  the  same  type  (on-center).  No 
inhibition  is  directly  transmitted  but  rather  is  supplied  by  inhibitor' 
interneurons.  The  profiles  would  remain  essentially  isotropic  and 
unchanged  in  shape  except  for  stronger  inhibition  since  the  lateral 
inhibition  is  applied  on  the  afferent  side  rather  than  on  the  efferent 
side.  At  the  cortical  level  the  geniculate  input  is  probably  exclusively 
excitatory  and  supragranular  pyramidal  cells  probably  receive  direct 
monosynaptic  contact  from  very  few  geniculate  axons.  Inhibition  is 
supplied  by  di-synaptic  or  polysynaptic  circuits.  Since  the  intra- 
cortical  inhibition  is  polysynaptic,  the  inhibitory  subfields  are 
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larger  than  the  excitatory  central  zone  as  well  as  being  slightly 
displaced.  The  inhibitory  side  lobes  although  silent  could  generate 
orientation  selectivity  and  the  width  of  the  excitatory  central  zone 
could  determine  the  optimal  size  or  spatial  frequency  tuning  for  the 
cell.  A  second  operation  of  lateral  inhibition  is  suggested  by  the 
recurrent  collaterals  of  pyramidal  cells  as  are  found  elsewhere  in 
the  neocortex  (Marin-Padil la ,  1969;  1970)  and  by  the  difficulty  in 
generating  responses  by  electrical  stimulation  in  the  optic  radiation 
below  the  cortex.  Antidromic  action  potentials  would  serve  to  silence 
cellular  activity  by  recurrent  inhibition.  Such  pericolumnar  inhibition 
as  Mountcastle  (1978)  has  described  it  would  by  the  sequential  regularity 
of  the  orientation  columns  be  directed  orthogonal  to  the  preferred 
orientation.  Formally  in  terms  of  the  simple  theoretical  receptive 

field  profile  described  by  Equation  5.2,  the  Laplacian  operator  takes 

2 

the  form  v  if  for  convenient  we  let  the  y-axis  be  the  axis  of 
x 

preferred  orientation.  When  the  operator  is  applied  the  equation  for 
the  receptive  field  profile  becomes 

r  2 

v'  A  •  -  8r‘'<c°sa)'  *  ^sucs.)  _  i8;2-  i6)  ♦  £„  ]e'<j)  (5.3) 

x  a  a  a  a  a 

The  receptive  field  profile  corresponding  to  Equation  5.3  is  shown 
in  Fig.  6.  The  pericolumnar  inhibition  would  have  the  effect  of 
sharpening  orientation  and  frequency  tuning. 

ii)  Local  and  global  population  parameters 

At  a  given  point  in  the  cortex  a  typical  receptive  field 
can  be  described  by  an  excitation  profile  such  as  that  given  by 
Equation  5.3.  Such  a  receptive  field  then  acts  as  a  local  spatial 
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filter  with  a  particular  center  frequency  and  frequency  bandwidth  and 
preferred  orientation  and  orientation  bandwidth.  Locally  the  center 
frequency  is  distributed  as  illustrated  in  Fig.  5,  and  the  frequency 
bandwidths  at  half  height  are  distributed  in  a  small  range  between 
1  and  2  octaves.  Depending  upon  location  the  orientation  preferences 
have  distributions  similar  to  those  shown  in  Fig.  4,  and  the 
orientation  bandwidths  have  a  distribution  similar  to  that  in  Fig.  3  . 
More  globally  the  distributions  of  orientation  preferences  and  center 
frequencies  change  with  retinal  eccentricity.  An  important  question 
is  the  degree  to  which  orientation  tuning  and  spatial  frequency 
selectivity  are  interrelated.  These  two  aspects  appear  to  be 
functionally  separable  since  orientation  tuning  bandwidth  for 
supragranular  neurons  (Mansfield  et  al,  1980)  or  presumed  supra- 
granular  neurons  (Schiller  et  al,  1976b)  does  not  appear  to  change 
with  retinal  eccentricity  at  least  up  to  20  degrees.  On  the  other 
hand  since  receptive  field  size  and  center  frequency  are  correlated! 
and  bandwidth  is  roughly  proportional  to  center  frequency  (1-2  octaves) ; 
and  center  frequency  shifts  with  retinal  eccentricity  (Schiller  et 
al,  1976c)  there  is  a  change  in  absolute  bandwidth  with  retinal 
eccentricity.  A  deeper  question  is  how  the  population  response 
profile  information  is  utilized  for  a  particular  task. 

iii)  Detection  mechanisms 

The  ensemble  of  supragranular  pyramidal  neurons  forms  an 
input  array  to  the  rest  of  the  visual  system  that  carries  the  necessary 
and  sufficient  information  for  optimal  acuity  performance.  If  l(x,y) 
denotes  the  luminance  distribution  in  the  retinal  image,  then  the  neural 
response  of  a  particular  class  of  orientation  selective  neurons  at  the 
corresponding  retinotopic  point,  N@  f(x,y)  is  given  by  the  equation 
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Ne,f{x,y)  =  s[l(x,y)]  (5.4) 

where  S  is  the  visual  system  operator.  The  operator  can  be  con¬ 
sidered  to  be  linear  if  the  input  to  the  neurons  is  exclusively 
from  X  cells  since  by  definition  they  respond  linearly.  The 
luminance  distribution  can  be  written  in  terms  of  the  Dirac 

delta  function,  6 ( x  -  £,  y  -  n),  as  follows 

00  00 

I(x»y)  -  f  f I ( S > t ) 6 ( x  -  £ ,y  -  n)d£dn  (5.5) 

Then  substituting  this  definition  of  I(x,y)  in  Equation  5.4  and 
using  the  linearity  of  the  S  operator  since  the  input  to  the 
supragranular  layers  is  dominated  by  X-cell  input,  we  obtain 

00  oo 

N0if(x,y)  =  /  /i(C,n)S[  6(x  -  e,y  -  n)  Idedn  (5.6) 

-00  -  00  w 

But  the  operation  of  S  on  the  delta  functions  is  simply  to  produce 
spatial  filter  functions  h(x,y;£,n,9)  which  correspond  to  the 
excitatory  profiles  previously  discussed  so  that 

00  00 

N8ff(x.y)  =  /  /I<£,n)h(x  -  e,y  -  n,5  ,n,e)d£dn  (5.7) 

-  00  -  00 

That  is  to  say  the  response  of  the  neurons,  the  integrated  discharge, 
is  the  convolution  of  the  luminance  distribution  with  the  excitatory 
profile  of  the  receptive  field.  Such  detailed  considerations 
based  upon  the  present  limited  knowledge  of  synaptic  circuitry  in 
the  central  visual  system  cannot  be  extended  with  confidence  beyond 
the  striate  cortex.  How  then  can  visual  behavior  be  dealt  with  in 
a  predictive  quantitative  manner;  by  choosing  a  visual  task  that 
requires  optimal  performance  such  as  visual  acuity  and  by  considerincr 
what  an  optimal  information  processing  system  can  do  given  the 
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output  array  from  VI.  As  will  be  seen  optimal  or  near  optimal 
detection  can  be  carried  out  by  at  least  two  neurally  realizable 
mechanisms  shown  schematically  in  Fig.  7. 

Insert  Fig.  7  about  here 

In  discussing  the  detection  process  involved  in  visual  acuity 
we  need  to  consider  the  issues  of  the  independence  of  noise  in 
different  neural  channels,  the  coherence  or  incoherence  of  the 
detection  and  the  nature  of  the  optimal  detection  mechanism.  An 
important  theorem  of  stochastic  processes  (Cinlar,  1972)  states  that 
the  sum  of  point  processes  that  are  Poisson  is  itself  Poisson.  For 
this  reason,  the  simplest  random  process,  a  Poisson  process  is 
a  good  first  approximation  to  the  stochastic  nature  of  impulse 
trains  for  cortical  neurons  when  suitably  corrected  for  refractoriness 
which  is  intrinsic  to  neural  activity  (Teich  and  McGill,  1976;  Teich, 
Matin  and  Cantor,  1978).  On  the  other  hand,  the  very  multiplicity  of 
sources  guarantees  the  statistical  independence  of  the  noise  in 
different  channels,  i.e.  the  noise  is  uncorrelated.  On  the  issue  of 
coherence  of  the  detection  process,  the  question  is  whether  phase 
information  between  channels  tuned  to  the  same  spatial  frequency  but 
at  different  retinotopic  locations  is  preserved.  With  a  coherent 
detector  spatial  frequency  bandwidth  and  spatial  summation  area  are 
inversely  related  but  with  an  incoherent  detector  the  two  are  independent 
The  studies  of  receptive  fields  in  the  more  rostral  portions  of  the 
visual  system  indicate  that  the  fields  are  large  but  in  general  do 
not  show  correspondingly  specific  spatial  frequency  tuning  so  incoherent 
detection  is  the  most  likely  form.  In  order  to  detect  a  pattern  of 
interest  presented  to  the  retina  the  central  mechanisms  can  be  considered 
to  monitor  the  output  of  M  orthogonal  channels  and  perform  a  decision 
process  that  is  independent  of  spatial  phase  except  locally.  Optimal 
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detection  arises  from  the  application  of  the  Neyman-Pearson  criterion 
which  maximizes  the  probability  of  detection  with  the  probability 
of  a  false  alarm  constrained  to  a  particular  value  as  described  by 
the  theory  of  signal  detectability  (Peterson,  Birdsall  and  Fox, 

1954;  Green  and  Swets,  1966).  The  general  decision 

rule  can  be  stated  in  terms  of  likelihood  ratios  but  for  a  Poisson 
noise-counting  distribution  the  decision  rule  reduces  to  a  simple 
comparison  of  a  pulse  count  to  a  criterion  number  of  pulses,  a 
threshold  process  readily  realizable  in  terms  of  known  neural  mechanisms. 
In  the  case  of  small  signals,  the  optimal  detector  is  closely  approx¬ 
imated  by  a  multi-band  detector,  i.e.  one  that  simply  compares  the 
sum  of  the  outputs  of  the  M  channels  to  a  criterion  level  (Cohn,  1978). 
Moreover,  the  optimal  detector  is  also  closely  approximated  by 
another  suboptimal  detector,  one  in  which  detection  occurs  if  the 
activity  in  at  least  one  channel  exceeds  a  threshold  (Nolte  and 
Jaarsma,  1967;  Green  and  Weber,  1980).  Such  a  probability  summation 
detector  is  readily  realizable  as  a  neural  OR-gate  and  is  computationally 
particularly  convenient.  The  detection  process  in  a  visual  acuity 
task  then  can  be  conceptualized  in  terms  of  a  set  of  orthogonal  spatial 
filters  with  uncorrelated  Poisson  noise  followed  by  an  incoherent  but 
quasi-optimal  detector. 

IV.  Behavioral  Experiments 

Behavioral  performance  reflects  organized  cellular,  mainly 
neural,  activity  and  for  a  detailed  understanding  requires  precise 
information  about  the  underlying  neural  activity.  However,  optimal 
performance  can  be  considered  within  the  context  of  a  computational 
theory  without  complete  information  of  all  neural  activity  but  rather 
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only  the  neural  activity  that  sets  constraints  on  optimality.  In 
the  case  of  simple  visual  acuity  optimal  performance  is  constrained  by 
the  spatial  filtering  carried  out  by  the  cortical  receptive  fields  in 
the  supragranular  layers  of  VI.  A  comparison  can  be  made  between 
experimentally  determined  spatial  visual  acuity  and  the  predictions 
of  a  computational  theory  based  on  an  optimal  incoherent  detector 
constrained  by  M  linear  orthogonal  spatial  filters  having  characteristics 
like  the  cortical  receptive  fields  for  a  particular  set  of  stimulus 
conditions.  However,  there  is  a  developing  consensus  that  such  a 
computational  theory  can  be  extended  to  a  wide  class  of  stimuli. 

A.  Comparisons  under  stabilized  viewing 

The  close  correspondence  in  behavioral  capacity  among  Old 
World  anthropoids  makes  possible  the  comparison  of  visual  performance 
in  man  with  predictions  from  a  computational  theory  based  upon  neural 
parameters  derived  mainly  from  the  macaque  monkey.  Visual  acuity 
assessed  by  contrast  sensitivity  functions  is  essentially  identical  in 
man  and  macaque  monkey  when  measured  under  the  same  stimulus  conditions 
(DeValois,  Morgan,  and  Snodderly,  1974).  For  oriented  grating  targets 
the  cross-species  similarity  assumption  for  neural  mechanisms  is  quite 
plausible  but  the  experimental  conditions  under  which  the  neural  data 
were  collected  must  be  carefully  considered. 

Precise  assessment  of  single  unit  receptive  fields  necessitated  the 
control'  of  eye  movements  typically  by  immobilizing  the  ocular  and  skel¬ 
etal  musculature  with  a  curare-like  drug,  gallamine  triethiodide  (Schil¬ 
ler  et  al,  1976a;  Mansfield  £t  al^  1980) .  Even  with  drug-induced 
paralysis  of  the  neuromuscular  synaptic  transmission  there  is  some 
residual  eye  movement  but  it  is  on  the  order  of  only  0.1°  over  the 
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course  of  10  minutes.  Similar  drug-induced  paralysis  in  human 
observers  results  in  a  reduction  in  apparent  contrast  so  that  large 
uniformly  illuminated  objects  disappear  and  edges  fade  (Stevens, 
Emerson,  Gerstein,  Kallos,  Neufeld,  Nichols  and  Rosenquist,  1976), 
a  phenomenon  observed  in  normal  vision  for  peripheral  targets  viewed 
with  steady  eccentric  fixation  under  dim  illumination.  Such  Troxler 
fading  is  confined  to  the  periphery  since  the  small  eye  movements 
present  in  steady  fixation  are  small  relative  to  the  dimensions  of 
the  receptive  fields  engaged  by  the  stimulus.  Steady  fixation  with 
foveal  viewing  does  not  achieve  the  same  result  since  even  in  the  best 
normal  observers  the  direction  of  gaze  about  a  fixated  target  is 
erratic  having  a  mean  radius  of  about  0.1°  and  microsaccades  are  not 
suppressed  (Higgins,  Daugman,  and  Mansfield,  1980) .  In  order  to  ex¬ 
amine  visual  performance  under  comparable  conditions  to  the  neuro¬ 
physiological  experiments  in  "any  detailed  psychophysical  task  would 
have  required  hours  of  total  paralysis  -  a  very  unrealistic  require¬ 
ment"  (Stevens  et  al,  1976).  An  alternate  means  of  achieving  retinal 
image  stabilization  is  to  compensate  optically  for  image  motion  on  the 
surface  of  the  retina. 

The  conventional  approach  developed  twenty  five  years  ago  involves 
the  use  of  a  contact  lens  either  with  a  mirror  attached  to  form  one 
arm  of  an  optical  lever  (Riggs,  Ratliff,  Cornsweet  and  Cornsweet,  1953) 
or  to  carry  a  miniature  projector  (Ditchburn  and  Pritchard,  1956). 

These  methods,  however,  are  limited  both  in  the  range  of  stimuli  that 
can  be  presented  and  in  the  ease  with  which  extended  and  detailed  ob¬ 
servations  can  be  made  in  several  observers  because  of  the  specially 
fitted  scheral  contact  lenses  involved.  Recent  innovations  in  the 
methods  of  accurate  eyetracking  which  do  not  require  physical  contact 
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have  set  the  stage  for  a  new  technique  of  retinal  image  stabilization. 

The  approach  uses  infrared  light  reflected  from  the  eye  to  determine  eye 
position.  As  shown  schematically  in  Fig.  8  the  novel  feature  is  that  the 

Insert  Fig.  8  about  here 

SRI  Dual  Purkinje  Image  Eyetracker  monitors  infrared  reflections  from  both  the 
cornea  (1st  Purkinje  image)  and  the  back  of  the  lens  (4th  Purkinje  image) 
(Cornsweet  and  Crane,  1973;  Clark,  1975) .  By  measuring  the  optical  distance 
between  the  two  images,  the  eyetracker  achieves  an  accuracy  of  greater  than 
1'  of  arc.  Such  a  high  degree  of  accuracy  suggested  to  B.  R.  Wooten  of  Brown 
University  the  possibility  of  developing  a  means  of  retinal  image  stabilization 
without  contact  lenses.  In  the  Wooten  modification  a  pair  of  fast  response 
(1-2  msec  risetime)  galvanometer  driven  mirrors  use  the  horizontal  and  vertical 
eye  movement  signals  of  the  eyetracker  to  compensate  optically  for  movement 
of  the  stimulus  across  the  retina  resulting  from  eye  movements. 

Figure  9  shows  an  observer  in  the  apparatus  and  a  typical  stimulus  pattern 

Insert  Fig.  9  about  here 

on  the  display  oscilloscope.  The  observer  views  the  target  monocularly  and 
holds  his  head  steady  by  means  of  a  hard  wax  bite  bar.  Typically  mydriatic  and 
cycloplegic  drugs  are  used  to  maintain  the  pupil  at  maximal  dilation  and  the 
lens  in  a  relaxed  state  of  accommodation.  A  television-like  raster  scan  on  the 
face  of  the  display  oscilloscope  can  be  readily  produced  by  waveform  generators 
to  yield  a  uniformly  illuminated  surface.  The  voltage  on  the  Z-axis  of  the 
oscilloscope  can  be  modulated  in  phase  with  each  frame  of  the  scan  to  produce 
patterns  on  the  display  face  using  electronic  hardward  for  simple  sinusoidal 
patterns  or  digital  to  analog  read-out  from  a  computer  for  more  complex  pattern!, 
The  choice  of  a  target  stimulus  with  which  to  probe  the  visual  sys¬ 


tem  and  test  the  range  of  validity  of  the  computational  theory 
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outlined  above  depends  upon  several  characteristics  of  the  primate 
visual  system.  By  using  low  contrast  patterns  the  response  remains 
in  the  linear  range.  By  choosing  patterns  with  a  small  spatial  ex¬ 
tent  a  piece-wise  isoplanatic  approximation  can  be  made  to  retinal 
inhomogeneity.  A  sinusoidally-modulated  Luminance  orating  such  as 
that  shown  in  Fig.  9B  is  easily  generated  and  controlled  and  can 
be  used  to  determine  the  frequency  response  which  is  related  to 
the  line-spread  function  by  a  Fourier  transform.  The  case  of  foveal 
vision  is  of  particular  interest  and  Fig.  10  shows  three  results 

Insert  Fig.  10  about  here 

obtained  for  a  small  grating  patch  centered  at  the  fovea  (±  1°) 
surrounded  by  a  broad  border  of  approximately  equal  luminance  to 
eliminate  edge  effects  which  could  swamp  the  response  of  interest. 
Temporal  transients  also  are  avoided  by  the  use  of  a  relatively  long 
display  period. 

The  basic  datum  consists  of  an  observer’s  contrast  threshold 
setting  but  each  data  point  is  typically  based  on  several  observations 
to  obtain  an  accurate  average.  Fig.  10A  displays  detection  threshold 
as  a  function  of  target  orientation  for  a  grating  of  relatively  high 
spatial  frequency.  The  solid  line  drawn  through  the  data  points  is  the 
prediction  from  the  computational  theory  scaled  to  the  sensitivity 
level  of  the  observer.  The  results  show  how  the  orientation  anisotropy 
present  in  the  neuronal  population  translates  into  a  difference  in  the 
detectability  of  targets  at  different  orientations.  These  results 
are  generalized  in  Fig.  10B  which  shows  detection  threshold  as  a 
function  of  spatial  frequency  of  the  grating  target,  for  l>oth  a  vertical 
and  an  oblique  angle  (45°).  The  solid  line  is  again  the  theoretical 
prediction  from  computation.  ‘At  low  spatial  frequencies  the  sensitivity 
is  depressed  beyond  that  inherent  in  the  frequency  response  of  the 
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central  fovea  because  the  limited  number  of  grating  cycles  present 
in  the  small  2°  target  decreases  the  optimization  of  detection  pro¬ 
duced  by  the  information  pooling  that  can  take  place  when  multiple 
bars  are  present.  However,  the  convergence  of  the  vertical  and 
oblique  sensitivities  reflects  the  dominance  of  the  response  by 
the  Y  cells  which  do  not  exhibit  orientation  anisotropy.  The 
effect  on  detection  threshold  of  prior  exposure  to  a  high  contrast 
grating  target  is  shown  in  Fig.  IOC.  The  solid  line  represents  the 
theoretical  curve  generated  from  the  computational  theory  by  taking 
the  difference  in  the  pooled  neural  response  before  and  after 
adaptation.  The  selective  elevation  of  threshold  reflects  the 
orientation  selective  nature  of  the  pattern  adaptation.  Since 
the  pooling  operation  in  effect  averages  over  the  spectrum  of 
orientation  tuning  curves  the  resulting  curve  resembles  the  tuning 
curve  of  the  average  receptive  field. 

B.  Implications  for  pattern  detection 

There  is  accumulating  evidence  that  the  detection  of  a 
wide  class  of  monochromatic  patterned  stimuli  can  be  explained  by 
a  computational  theory  based  on  quasi-optimal  incoherent  detection 
applied  to  the  output  of  an  array  of  independent  channels  whose 
medium  bandwidth  filter  characteristics  resemble  the  receptive 
field  excitatory  profiles  of  efferent  neurons  in  the  supragranular 
layers  of  VI.  Incoherent  detection  is  consistent  with  the  localized 
effects  of  pattern  adaptation.  Previous  work  from  this  laboratory 
(Legge,  1976)  demonstrated  that  the  adaptation  is  retinotopically 
localized.  Legge  found  that  adaptation  to  a  high  contrast  fine  line 
did  not  affect  subsequent  detection  of  spatially  extended  sinusoidally 
modulated  grating  patterns  as  would  be  predicted  if  the  effect  of 
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adaptation  were  local  rather  than  global.  Daugman  and  Mansfield  (1979) 
found  that  the  adaptation  is  not  even  transferred  interhemispher- 
lcally  since  no  adaptation  is  produced  by  an  eccentric  high  contrast 
grating  patch  placed  in  the  homologous  location  in  the  opposite 
hemifieid.  Graham  and  her  colleaques  (Graham  and  Nachmias,  1971; 

Graham,  Robson  and  Nachmias,  1978)  have  demonstrated  independence 
of  detection  of  separate  components  in  a  compound  grating  even 
when  presented  to  a  relatively  homogeneous  retinal  region,  a  result 
consistent  with  the  existence  of  independent  channels.  Variation 
with  retinal  eccentricity  of  several  measures  such  as  visual  acuity 
(Aulhorn  and  Harms,  1972),  summation  area  (Westheimer,  1967),  Mach 
band  locations  (Shipley  and  Wier,  1972),  contrast  sensitivity  functions 
(Hilz  and  Cavonius,  1974),  and  line  spread  functions  (Hines,  1976; 

Limb  and  Rubinstein,  1977;  Wilson,  1978)  are  consistent  with  an  increase 
in  receptive  field  size  with  distance  from  the  fixation  point.  The 
work  from  Wilson's  laboratory  in  particular  suggests  that  the  increase 
is  linear  at  least  up  to  4°.  The  scatter  in  receptive  field  sizes 
and  types  at  a  given  retinotopic  projection  point  is  reflected  in  the 
differences  in  detectability  of  different  patterns  presented  at  the 
same  retinal  locus  (Kulikowski  and  Kinq-Smith,  1973;  Wilson  and  Geise, 
1977).  Experiments  with  frequency  gradient  patterns  (Wilson  and 
Geise,  1977)  and  with  artificial  central  scotomas  (Kelly,  1978)  indicate 
that  all  receptive  field  sizes  are  represented  in  the  foveal  projection 
region  but  that  there  is  a  decline  in  the  relative  proportion  of  small 
receptive  fields  with  retinal  eccentricity.  On  the  important  issue 
of  the  estimate  of  the  spatial  frequency  bandwidth  of  the  receptive 
fields  a  growing  consensus  indicates  that  althouqh  several  studies  have 
been  interpreted  as  implying  extremely  narrow  bandwidths  (1/4  octave) 
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(Sachs,  Nachmias  and  Robson,  1971;  Kulikowski  and  King-Smith,  1973; 
Quick  and  Riechert ,  1973)  subsequent  work  has  shown  that  if  an  inco¬ 
herent  detection  process  such  as  probability  summation  across  space 
is  taken  into  account  the  data  are  consistent  with  medium  bandwidths 
(Mostafavi  and  Sakrison,  1976;  Stromeyer  and  Klein,  1975;  Graham 
and  Rogowitz,  1976;  King-Smith  and  Kulikowski,  1975;  Wilson,  1978; 
Wilson  and  Bergen,  1979;  Bergen,  Wilson  and  Cowan,  1979) .  The  band¬ 
width  estimates  range  from  1.5  to  2.1  octaves  at  half-height.  The 
data  of  Blakemore  and  Campbell  (1969)  obtained  using  spatially  sel¬ 
ective  frequency  adaptation  yields  an  estimate  of  1.5  octaves  full 
bandwidth  at  half-height  but  retinal  inhomogeneity  and  probability 
summation  are  not  taken  into  account  resulting  in  an  underestimation. 
Mostafavi  and  Sakrison  (1976)  obtained  an  estimate  of  2.1  octaves  using 
band-limited  noise  displays  taking  into  account  retinal  inhomogeneity 
and  spatial  probability  summation  but  not  summation  over  channels 
resulting  in  an  overestimation.  Wilson  and  his  colleagues  (Wilson, 
1978;  Wilson  and  Bergen,  1979;  Bergen,  Wilson,  and  Cowan,  1979)  us¬ 
ing  subthreshold  summation  to  determine  the  line-spread  function 
found  1.75  octaves  taking  into  account  spatial  probability  summation 
over  channels  and  space.  The  monkey  neurophysiology  data  as  described 
earlier  yields  estimates  of  1  -  2  octaves  (Schiller  et  a_l,  1976c; 

Poggio  e_t  al ,  1977  ;  DeValois  et  al_,  1978).  De  Valois  et  al's  measure¬ 
ments  made  under  conditions  comparable  to  the  psychophysical  measure¬ 
ments  gave  a  median  value  of  1.4  octaves  and  1.2  octaves  for  the 
more  orientation  selective  neurons.  A  similar  situation  holds  for 
orientation  selectivity.  Early  studies  particularly  those  that  em¬ 
ployed  subthreshold  summation,  e.g.  Kulikowski,  Abadi  and  King-Smith 
(197!)  were  interpreted  as  evidence  for  extremely  narrow  orientation 
bandwidths.  e.g.  ±  3°.  other  measurement  procedures  using  adaptation 


Mansfield  Primate  Striate  Cortex -35 

(Blakemore  and  Nachmias,  1971)  and  masking  (Campbell  and  Kulikowski, 
1966)  gave  estimates  of  ±7°  to  +  15°.  However,  these  studies  did 
not  take  into  account  spatial  probability  summation.  By  taking 
spatial  probability  summation  into  account  the  data  from  subthres¬ 
hold  summation  experiments  are  consistent  with  orientation  band- 
widths  on  the  order  of  ±  16°.  Mostafavi  and  Sakrison  (1976)  using 
two  dimensional  filtered  noise  and  assuming  a  Gaussian  shape,  retinal 
inhomogeneity  and  a  spatial  probability  summation  index  of  6  obtained 
an  orientation  bandwidth  of  ±  14.4°;  however,  they  did  not  take  into 
account  summation  across  channels  or  orientation  anisotropy.  As 
we  have  seen  taking  into  account  spatial  probability  summation, 
orientation  anisotropy  and  probability  summation  across  channels 
yields  an  estimate  of  ±  20°  (see  Fig.  10C) .  The  neurophysiological 
data  uniformly  concur  on  an  estimate  of  average  bandwidth  of  ±20° 
to  ±  25°  (Schiller  et  al ,  1976b;  Poggio  et  al,  1977;  DeValois  et  al , 
1973;  Mansfield  et  al ,  1980).  The  concordance  of  results  from 
behavioral  and  neurophysiological  studies  particularly  regarding 
orientation  and  frequency  bandwidths  carries  a  number  of  implications 
for  visual  processing.  For  example,  the  view  that  the  visual  system 
performs  some  type  of  Fourier  analysis  on  a  visual  scene  (Campbell 
and  Robson,  1968;  Blakemore  and  Campbell,  1969;  Pollen,  Lee  and 
Taylor,  1971)  is  oversimplified  at  best  since  the  channel  bandwidths 
are  too  broad.  The  apparent  sharp  tuning  is  the  result  of  the  statis¬ 
tical  nature  of  the  incoherent  detection  process.  On  the  other 
hand  an  alternative  view  consistent  with  experimental  facts  reviewed 
here  is  that  the  visual  system  analyzes  target  objects  using  an 
ensemble  of  medium  bandwidth  spatial  filters  whose  characteristics 
match  those  of  the  receptive  fields  of  supragranular  striate 


cortical  neurons. 
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V.  Future  Prospects 

The  development  of  a  truly  computational  theory  of  primate 
vision  is  an  exciting  prospect  but  one  that  can  be  realistically 
approached.  Two  guiding  principles  have  emerged  from  the  studies 
reviewed  here.  First  the  primate  visual  system  is  organized  in 
terms  of  functionally  significant  channels.  Secondly  optimal  visual 
performance  can  be  analyzed  into  a  critical  transformation  site 
and  optimal  utilization  of  the  information  from  that  site.  The 
filtering  operations  performed  by  the  visual  system  up  to  the  level 
of  the  striate  cortex  and  by  the  striate  cortex  itself  give  rise 
to  a  filtered  neural  image  of  the  retinal  stimulus  represented  by 
the  population  response  profile  of  the  efferent  neurons,  the  trans¬ 
cortical  projection  of  which  consists  of  the  pyramidal  cells  in 
the  supragranular  layers.  This  neural  image  contains  all  the  fine 
grain  information  available  to  the  more  rostral  portions  of  the 
visual  system  which  operate  upon  the  information  in  a  manner  similar 
to  a  noncoherent  optimal  detector.  In  the  detailed  example  the 
stimuli  were  stationary,  low  contrast,  monochromatic ,  monocularly 
presented  targets  and  the  task  was  one  of  detection.  Each  of  these 
restrictions  can  be  lifted  or  modified. 

The  neurons  in  the  supragranular  layers  respond  differentially 
to  direction  of  target  motion,  but  no  quantitative  comparison  has 
been  made  with  the  directionally  sensitive  mechanisms  revealed  by 
prior  adaptation  in  psychophysical  experiments.  One  experimental 
finding  which  makes  reasonable  the  suggestion  that  these  neurons 
mediate  the  detection  of  target  direction  is  that  target  detection 
is  superior  to  detection  of  direction  of  motion  even  in  the  absence 
of  eye  movements  achieved  using  retinal  image  stabilization 
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(Mansfield  and  Nachmias,  1980),  a  result  consistent  with  the 
dominance  of  X-input  over  Y-input  into  these  layers.  Although 
the  simplest  cases  to  analyze  are  those  at  threshold,  linearity 
extends  about  that  level  (Mansfield,  1976a).  Indeed  as  Cannon 
(1979)  has  shown  the  entire  contrast  sensitivity  curve  remains 
parallel  to  those  generated  from  equal  apparent  contrast  up  to 
at  least  a  70%  contrast  ratio.  At  suprathreshold  levels  in  the 
linear  range,  the  spatial  filtering  imposed  by  the  striate  cortex 
has  predictable  but  not  intuitively  obvious  consequences  such  as 
the  distortions  from  veridicality  epitomized  by  the  familiar  op¬ 
tical  illusions  (Ginsburg,  1979)  or,  for  instance,  the  spatial 
distribution  of  eye  saccades  and  fixations  in  free  viewing  of  even 
a  complex  target  such  as  a  face  (Ginsburg,  Mansfield  and  Higgins, 
1980) .  Saccadic  eye  movements  also  induce  an  adaptive  suppression 
of  retinal  blur  by  masking  that  is  spatially  frequency  tuned  and 
can  be  modelled  in  terms  of  inhibitory  interactions  between  fast 
and  slow  responding  spatial  filters  (Breitmeyer  and  Ganz,  1976; 
Rogowitz,  1980).  Within  the  same  conceptual  framework,  binocular 
interactions  such  as  interocular  adaptation  and  masking  or  even 
local  stereopsis  can  be  usefully  analyzed  in  terms  of  striate 
neuronal  activity.  On  the  other  hand,  successful  extension  of  the 
approach  of  the  cellular  synthesis  of  behavior  described  here  to 
more  complex  visual  perceptual  and  visuomotor  behaviors  such  as 
global  stereopsis,  object  color  determination  and  object  tracking 
will  require  a  more  detailed  understanding  of  the  primate  visual 
system  particularly  of  regions  beyond  VI. 

The  basis  for  a  new  and  detailed  understanding  of  the 


primate  extrastriate  cortex  will  rest  on  three  developments  in 
cellular  and  behavioral  neurobiology.  First,  information  on 
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the  general  functional  subdivision  of  labor  among  the  different 

regions.  A  recurrent  theme  of  the  chapters  in  this  volume  is 

the  existence  of  visual  subsystems,  and  a  premier  goal  of 

behavior  analysis  must  be  that  of  fractionating  an  animal's 

repertoire  into  units  that  optimally  engage  a  particular 

subsystem.  Insights  into  causal  sequences  in  the  dynamic 

unfolding  of  a  behavioral  script  are  now  possible  with  single 

and  multi -unit  recording  from  alert  animals  in  well  defined 

behavioral  tasks  with  eye  movements  closely  monitored  (e.g. 

the  study  by  Lynch,  Mountcastle,  Talbot  and  Yin  (1977)  of 

the  initiation  and  control  of  visual  grasping  of  an  object  of 

motivational  significance) .  On  the  other  hand,  anatomical 

labelling  of  a  subsystem  by  its  elevated  metabolic  activity  when 

3  14 

engaged  can  be  accomplished  by  using  H-  or  C-2  deoxyglucose 
which  accumulates  in  cells  since  it  does  not  substitute  for 
glucose  beyond  a  certain  point  in  the  Krebs  cycle  (for  example, 
Frost  (this  volume)  using  this  technique  has  localized  the 
region  of  a  pigeon's  brain  dealing  with  relative  motion). 

In  the  living  animal  positron  emission  transverse  tomography 
will  soon  both  supplement  the  resolution  of  metabolic  activity 
on  the  cortical  surface  now  possible  with  gamma  emission  and 
provide  resolution  in  three  dimensions  eventually  down  to  the 
millimeter  level.  Secondly,  an  extensive  knowledge  of  the 
interconnections  between  cortical  regions  and  the  interactions 
with  subcortical  structures.  Area  VI  is  the  simplest  visual 
cortical  area  in  terms  of  its  connections  and  yet  it  is 
reciprocally  interconnected  with  at  least  four  other  structures: 
dLGN,  the  pulvinar  complex,  and  Areas  VI  and  MT.  Although 


1 


Mansfield 


Primate  Striate  Cortex-39 


connections  can  sometimes  be  inferred  from  disconnection 
syndromes  produced  by  lesions,  such  an  approach,  applied  to  a 
distributed  processing  system  such  as  the  visual  system, 
particularly  in  advance  of  anatomical  investigations,  can 
frequently  yield  paradoxical  results  (see  chapters  by  Dean 
and  by  Levine  for  a  discussion  of  such  inferences  for  infero- 
temporal  lesions) .  Fortunately,  recent  advances  in  neuroanatomical 
labelling  by  axonal  transport  have  brought  this  question  within 
the  range  of  experimental  analysis  so  that  precise  information  is 
accumulating  (see  Van  Essen  (1979)  for  an  excellent  recent 
review) .  Since  the  function  played  by  a  particular  region  is 
determined  not  only  by  the  intrinsic  operations  it  performs 
but  by  the  pattern  of  its  afferents  and  efferents  in  the 
circuit  of  which  it  forms  a  part,  even  a  first  approximation 
to  the  necessary  circuit  diagram  such  as  the  one  illustrated 
in  Fig.  1A  provides  a  rational  basis  for  identifying  functional 
subsystems  (see  the  chapter  by  Ungerleider  and  Mishkin  discussing 
two  cortical  visual  subsystems:  one  terminated  by  the  infero- 
temporal  region  TE  and  the  second  by  the  inferior  parietal 
lobule,  area  PG) .  Thirdly,  an  understanding  at  the  synaptic 
level  of  the  functioning  of  cortical  columns,  the  modules  of 
organization  of  the  neocortex.  Although  a  connectional 
diagram  even  as  detailed  as  Fig.  IB  together  with  recording  and 
lesion  studies  can  illuminate  the  role  of  a  particular  region, 
for  example  that  of  the  striate  cortex  as  a  preprocessor  o^  object 
information  but  the  mechanimsm  by  which  the  cortex  operates 
upon  its  afferent  input  requires  a  more  fine  grain  analysis. 

As  Mountcastle  (1976)  points  out:  "the  central  problem  of  the 
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intrinsic  physiology  of  the  cerebral  cortex  is  to  discover 
the  nature  of  the  neuronal  processing  within  the  translaminar 
chains  of  interconnected  cells  (in  columns)."  Techniques  are 
now  available  to  determine  the  synaptic  circuitry  of  morpho¬ 
logically  identified  neurons  in  functional  modules  of  neocortex 
using  in  vitro  recording  from  brain  slices  (Mansfield  and 
Simmons,  1979;  Mansfield,  in  preparation).  Anatomical  and 
physiological  evidence  (see  review  by  Mountcastle,  1978) 
suggests  that  cortical  modules  may  have  a  unitary  organization 
so  that  detailed  knowledge  of  the  synaptic  organization  in  one 
cortical  region  could  serve  as  a  paradigm  for  the  cortex  as 
a  whole.  The  present  and  continuing  challenge  will  be  to 
synthesize  the  results  from  these  three  lines  of  development 
into  a  coherent  computational  theory  of  visual  behavior. 
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Figure  Captions 

Figure  1.  Anatomical  organization  of  the  primate  visual  system. 

Nomenclature:  R  is  the  retina;  X,  Y,  W  designate  the  axons 

of  three  classes  of  ganglion  cells  or  at  the  geniculate  the 
projections  to  and  from  the  parvocel lular  and  magnocellular 
layers,  see  Fig.  IB;  DLGN  is  the  dorsal  lateral  geniculate 
nucleus;  PT  is  the  pretectum;  SC  is  the  superior  colliculus; 

PL  is  the  lateral  pulvinar;  PI  is  the  inferior  pulvinar; 

VI  is  the  striate  cortex;  V2  etc.  are  the  prestriate  visual 
areas  (See  Van  Essen,  1979) ;  IT  is  the  inferotemporal  cortex 
(cytoarchitectonic  areas  TEO  and  TE  of  von  Bonin  and  Bailey) ; 
FEF  are  the  frontal  eye  fields  in  the  region  of  the  arcuate 
sulcus  consisting  of  cytoarchitectonic  areas  8A,  8B  and 
posterior  9  in  the  terminology  of  Brodmann;  MT  is  a  region 
at  the  posterior  end  of  the  temporal  sulcus  described  as 
the  visual  area  of  the  superior  temporal  sulcus  by  Zeki  (1971) 
but  probably  homologous  to  the  middle  temporal  area  in  the 
owl  monkey;  IPS (OA)  is  a  region  delineated  by  Ungerleider 
and  Mishkin  (this  volume)  using  anterograde  transport  from 
VI  of  radioactive  amino  acids  in  area  OA  (von  Bonin  and  Bailey) 
at  the  depth  of  the  posterior  extent  of  the  intraparietal 
sulcus;  IPS (PG)  is  a  region  delineated  by  Mesulam,  van  Hoesen, 
Pandya  and  Geschwind  (1977)  using  retrograde  transport  of 
horseradish  peroxidase  from  PG  and  is  located  in  the 
intraparietal  sulcus  anterior  to  IPS(OA);  TF  is  a  region 
described  by  von  Bonin  and  Bailey  on  the  medial  surface  of 
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the  temporal  lobe  adjacent  to  TE  and  OA;  IPL(PG)  is  the 
region  PG  described  by  von  Bonin  and  Bailey  on  the  inferior 
parietal  lobule  and  corresponds  approximately  to  the  posterior 
half  of  area  7  of  Brodmann. 

A.  Schematic  representation  of  the  main  structures  showing 

the  principal  pathways  with  arrows  indicating  the  direction 
of  flow  of  information  and  control.  For  clarity  of 
presentation  only  ipsilateral  connections  have  been  shown 
and  the  detailed  structure  of  the  limbic  and  motor  systems 
and  that  of  the  prestriate  visual  cortical  field  has  been 
omitted.  For  purposes  of  this  diagram  area  6aB,  premotor 
cortex,  which  receives  a  projection  from  area  9p  in  FEF 
is  included  in  the  motor  system  along  with  the  pons 
(which  receives  input  from  IPL(PG),  MT  and  intervening 
regions  (Glickstein,  Cohen,  Dixon,  Gibson,  Holland, 

Labossie  and  Farrel,  1980))  and  the  interstitial  nucleus 
of  Cajal  (which  receives  input  from  the  superior  colliculus) . 
Major  sources  of  evidence  for  the  newly  described  pro¬ 
jections  in  the  cortical  visual  field:  projections  to 
IPL(PG)  are  based  on  Mesulara  et  &1_.  (1977)  and  Divac, 

LaVail,  Rakic  and  Winston  (1977);  projections  to  and  from 
limbic  system  based  on  Nauta  and  Domesick  (1980) ;  pro¬ 
jection  from  motor  system  to  visual  cortical  field  based 
on  Johnston,  McKinney  and  Coyle  (1979);  projections  from 
FEF  (areas  8  and  9p)  based  on  Kunzle  and  Akert  (1977) 
and  Kunzle  (1978);  projection  to  IPS(OA)  based  on 
Ungerleider  and  Mishkin  (this  volume) ;  connections  to 
and  from  VI  based  on  Lund  et  al.  (1975)  .  The  dashed 
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rectangles  indicate  structures  whose  substructures  are 
not  indicated  or  not  known  in  detail,  e.g.  prestriate 
visual  areas.  Arrows  to  dashed  rectangles  indicate 
projections  to  more  than  one  substructure;  arrows  from 
such  rectangles  indicate  projections  from  more  than  one 
substructure . 

B.  Schematic  representation  of  the  main  structures 
in  the  retino-cortical  pathways  with  arrows  indicating 
the  direction  of  information  flow.  For  clarity  of 
presentation  only  the  connections  from  one  hemi-retina 
and  to  one  hemisphere  are  shown.  The  brackets  on  the 
left-hand  side  of  the  lateral  geniculate  nucleus  indicate 
feedback  distributed  over  the  parvocellular  (upper  bracket) 
or  magnocellular  (lower  bracket)  layers.  Major  sources 
of  evidence:  for  the  existence  and  distribution  of 
X,  Y  and  W  type  retinal  ganglion  cells  (DeMonasterio , 

1978a,  1978b;  Schiller  and  Malpeli,  1977);  for  the  pro¬ 
jections  from  retina  to  superior  colliculus  and  pretectum 
(Schiller  and  Malpeli,  1977;  Bunt,  Hendrickson,  Lund, 

Lund  and  Fuchs,  1975);  for  the  connections  from  retina  to 
dorsal  lateral  geniculate  nucleus  (Dreher,  Fukada  and 
Rodieck,  1976;  Schiller  and  Malpeli,  1978);  for  the  pro¬ 
jections  from  geniculate  to  striate  cortex  (Hubei  and 
Wiesel,  1972;  Rezak  and  Benevento,  1979);  for  the  pro¬ 
jections  from  striate  cortex  to  cortical  and  subcortical 
structures  (Lund  et.  aK  ,  1975;  Van  Essen,  1979;  Maunsell, 
Bixby  and  Van  Essen,  1979;  Ogren  and  Hendrickson,  1976; 
Rezak  and  Benevento,  1979) ;  for  feedback  connections  to 
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striate  cortex  (Van  Essen,  1979;  Ogren  and  Hendrickson, 
1977;  Wong-Riley,  1979);  for  connections  among  superior 
colliculus,  pretectum  and  their  projections  (Benevento 
and  Fallon,  1975;  Rezak  and  Benevento,  1977;  Berman, 
1977  (cat);  Benevento,  1980);  for  feedback  from  V2  and 
MT  to  geniculate  (Lin  and  Kaas,  1977  (owl  monkey)). 

C.  Schematic  diagram  of  anatomically  segregated  visual 
processing  in  striate  cortex  (after  Mansfield,  1979) . 
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Figure  2 


Orientation  selectivity  in  binocular  bidirectional 

striate  neuron  in  monkey.  (After  Mansfield  et  al_.  ,  198 

A.  Anatomical  reconstruction  of  microelectrode 
track.  On  the  left  side  of  the  diagram  the 
nomenclature  for  each  distinguishable  lamina 
of  the  striate  cortex  is  indicated.  The 
solid  black  circles  represent  the  locations 
of  4  marking  electrolytic  lesions.  The  short 
line  segments  perpendicular  to  the  solid  line 
marking  the  track  represent  the  recording  sites 
of  isolated  single  units.  The  solid  black  arrow 
indicates  the  recording  site  of  Unit  8T107 
described  more  fully  in  Figures  2B  and  2C. 

B.  Receptive  field  map  and  response  profile. 

The  solid  black  rectangle  represents  the 

minimal  discharge  receptive  field  of  the  neuron 

in  the  right  visual  hemifield.  The  receptive 

field  measured  1/2°  by  1°  and  was  located 

approximatley  6°  from  the  fovea.  Arranged 

around  the  receptive  field  map  are  peristimulus 

discharge  histograms  generated  by  a  long  slit 
o 

subtending  0.1  in  width  moved  through  the  dis- 

o 

charge  zone  at  a  velocity  of  5.6  /sec  at  the 
orientation  indicated.  Each  peristimulus 
histogram  is  based  upon  15  stimulus  presentations. 

C.  Orientation  tuning  curves.  Integrated  dis¬ 
charge  as  a  function  of  target  slit  orientation. 
Each  data  point  is  the  average  calculated  from 
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15  stimulus  presentations.  The  solid 
squares  represent  the  responses  from  the 
receptive  field  in  the  right  eye;  the  open 
circles  represent  the  responses  from  the 
receptive  field  in  the  left  eye.  The  error 
bars  represent  +  one  standard  error  of  the  mean 
integrated  discharge.  The  solid  lines  fitted 
to  the  data  points  represent  theoretical 
functions  for  orientation  tuning  derived  from 
a  simple  model  of  orientation  determination. 
(Modified  from  Mansfield,  Ronner  and  Daugman  (1980)). 

Figure  3.  Frequency  distribution  of  orientation  selectivity 
in  striate  neurons.  Orientation  selectivity  was 
measured  as  the  half-bandwidth  at  half-maximal 
response  on  an  orientation  tuning  curve  as  shown  in 
the  insert.  The  measurements  were  obtained  from 
96  independently  sampled  neurons.  (From  Mansfield 
et  al  (1930) ) . 

Figure  4.  Distribution  of  axes  of  orientation  preference  for 
striate  cortical  neurons.  Each  neuron  was  sampled 
from  a  single  penetration  with  multiple  penetrations 
separated  by  at  least  500  jjm  across  the  cortical 
surface. 

A.  Polar  histogram  of  distribution  for  receptive 
fields  of  independently  sampled  neurons  with 
foveal  receptive  fields.  (N=82).  The  deviation 
from  uniformity  is  statistically  significant 
(x2  =  15.94,  df  =  8;  p  <  0.05). 
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Figure  5 


Figure  6. 


B.  Polar  histogram  of  distribution  of  independently 

sampled  neurons  with  parafoveal  receptive 

fields  (N  =  128).  The  Chi-squared  test  for 

departure  from  a  uniform  distribution  was  not 

2 

significant  at  the  5%  level  (x  =  1.66, 
df  =  8;  p  >  0.5).  (From  Mansfield  et  al  (1980)). 
Frequency  distribution  of  optimal  spatial  frequency 
of  striate  cortical  neurons  in  the  foveal  and 
parafoveal  projection  regions.  The  curve  to  the 
left  represents  the  parafoveal  striate  neurons 
from  the  projection  region  4°-6°  from  the  fovea. 

The  curve  to  the  right  represents  the  foveal  striate 
neurons  from  the  projection  region  covering  the 
central  2°. 


Schematic  diagram  of  a  candidate  mechanism  for 

receptive  field  organization  of  supragranular  neurons. 

A.  Excitatory  profile  generated  by  retinal 
lateral  inhibition. 

B.  Candidate  neural  network  in  supragranular 
region  of  striate  cortex  exhibiting  recurrent 
lateral  inhibition. 

C.  Excitatory  profile  generated  by  operating  on 
profile  in  A.  by  neural  network  in  B. 


Figure  7.  Schematic  diagram  of  two  quasi-optimal  neurally 
realizable  pattern  detector  mechanisms. 
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Figure  8 


A.  Pooling  detector.  In  this  case  the  luminance 
corresponding  to  the  visual  stimulus  distribution  is 
processed  by  linear  spatial  filters  and  the  responses 
of  the  filters  summed  or  pooled  in  a  detector  which 
responds  if  the  combined  input  exceeds  some  criterion 
level.  For  clarity  of  presentation  the  independent 
noise  sources  for  the  spatial  filters  have  been 
omitted . 

B.  Probability  summation  detector.  In  this  case  the 
luminance  distribution  corresponding  to  the  visual 
stimulus  is  processed  by  linear  spatial  filters  but 
the  response  of  each  is  handled  by  an  independent 
detector  which  responds  if  the  input  exceeds  a  criterion 
level.  So  the  action  of  the  effector  depends  on  the 
action  of  one  or  more  of  the  detectors.  For  clarity 

of  presentation  the  independent  noise  sources  for  the 
spatial  filters  have  been  omitted. 

Schematic  diagram  of  Dual  Purkinje  Eyetracker  and  Retinal 
Image  Stabilization  System.  In  determining  eye  position, 
a  low  intensity  invisible  beam  of  infrared  light  from  a 
light  emitting  diode  is  reflected  from  the  eye  and  the 
images  of  the  first  Purkinje  image  (corneal  reflection)  and 
fourth  Purkinje  image  (posterior  surface  of  the  lens)  are 
separately  focussed  on  quadrature  photodetectors.  The 
error  signals  from  the  quadrature  photodetectors  drive 
mirrors  in  the  optical  path  via  servo  motors.  The  output 
of  the  second  quadrature  photodetector  (fourth  Purkinje 
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Figure  9. 


Figure  10. 


image)  then  corresponds  to  the  difference  between  the 
first  and  fourth  Purkinje  images,  a  measure  of  eye  position 
that  is  relatively  invariant  under  lateral  movement  of  the 
head.  For  retinal  image  stabilization  the  eye  position 
signal  is  used  to  operate  galvanometer  driven  mirrors  in 
the  optical  path  of  the  viewer  which  compensate  for  vertical 
and  horizontal  eye  movements  by  appropriate  movements  of 
the  visual  target,  a  computer-controlled  video  display  on 
a  CRT.  Independent  of  the  stabilization  a  set  of  visible 
LEDs  serve  as  fixation  targets.  In  addition  the  observer's 
responses  are  accessible  to  the  computer  and  can  be  used 
in  interactive  psychophysical  procedures. 

A.  Photograph  of  observer  viewing  target  under  retinal 
image  stabilization  by  using  the  outputs  of  the  SRI 
Dual  Purkinje  Image  Eyetracker  to  drive  small  mirrors 
on  the  Stimulus  Deflector  seen  in  the  foreground. 

B.  Photograph  of  target  on  CRT  at  high  contrast. 

Comparison  of  calculated  response  from  synthesized  striate 
neuron  population  with  human  psychophysical  data  on 
orientation  acuity.  The  theoretical  curves  have  been 
scaled  in  amplitude  to  fit  best  the  experimental  data. 

A.  Detection  threshold  as  a  function  of  target  orientation 
for  a  8  cycle/degree  sinusoidally  modulated  luminance 
grating  subtending  2°  foveally.  The  circles  represent 
the  data  points;  the  solid  line  represents  the  computed 
values . 

B.  Detection  threshold  as  a  function  of  grating  spatial 
frequency.  The  circles  represent  the  data  points  for 
the  vertical  target;  the  squares  represent  the  data 
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for  the  oblique  (45°)  target.  The  solid  lines  represent 
the  computed  values. 

C.  Detection  threshold  elevation  following  adaptation  to 
8  cycle  per  degree  grating  subtending  2°  foveally. 

The  squares  represent  the  data  obtained  with  adaptation 
and  testing  in  the  same  eye.  The  x's  represent  the 
data  obtained  with  testing  in  the  contralateral  eye. 

The  solid  curve  represents  the  computed  values  with 
adaptation  proportional  to  neural  response. 
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